Skip to content

Conversation

@pcrespov
Copy link
Member

@pcrespov pcrespov commented Sep 26, 2025

What do these changes do?

This PR introduces a new RPC client namespace configurable and restrict unintended cross-service calls. This is for now only used in the RPC communication between the api-server (client) and the wb-api-server (server)

Motivation

The simcore_service_webserver package is reused across many services (wb-api-server, wb-auth, wg-garbage-collector, wg-listener, …). This package defines an RPC server under a fixed namespace (WEBSERVER_RPC_NAMESPACE). See in the image below RPC server (4) and RPC clients (1,2,3):
image

Because all services share this namespace, a client can unintentionally reach any of them.

  • Example: api-server should only talk to wb-api-server, but may end up reaching webserver or others.

Changes

  • Introduced a configurable RPCClient that allows setting the RPC server namespace.
  • Adjusted client-service mappings using WEBSERVER_RPC_NAMESPACE in the app settings:
    • api-serverwb-api-server
    • payments & directorv2webserver (stills use the old method)
  • Disabled the RPC server for services that should not expose it (e.g., wb-auth, wg-garbage-collector, wg-listener).
  • Added a new RPCClient implementation under:
    packages/service-library/src/servicelib/rabbitmq/rpc_interfaces/webserver/v1

Next Steps

Related issue/s

How to test

  • 🚨 @bisgaard-itis can you pls add here some steps on how you manually tested it? thx

Update from @bisgaard-itis:

I tested this by running the stack locally and checking that I can run functions via the api-server function interface. I.e. the first call to trigger the celery tasks. Then I scaled the wb-api-server to 0 replicas in portainer and checked, that the function interface was completely broken. Then I did the opposite: Scaled the wb-api-server back to 1 and scaled the web-server to 0. With this, the frontend of course was broken and I checked that I could still run functions via the api-server. This proves that the api-server now sends its RPC requests to the wb-api-server and not the webserver.

Dev-ops

  • No new env var required since we reused WB_API_WEBSERVER_HOST for WEBSERVER_RPC_NAMESPACE: ${WB_API_WEBSERVER_HOST}

@pcrespov pcrespov added this to the Cheops milestone Sep 26, 2025
@pcrespov pcrespov added a:webserver webserver's codebase. Assigning the area is particularly useful for bugs a:services-library issues on packages/service-libs t:maintenance Some planned maintenance work a:apiserver api-server service labels Sep 26, 2025
@codecov
Copy link

codecov bot commented Sep 26, 2025

Codecov Report

❌ Patch coverage is 27.65152% with 191 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.72%. Comparing base (27ff515) to head (cbf77c2).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8431      +/-   ##
==========================================
- Coverage   87.94%   87.72%   -0.23%     
==========================================
  Files        1976     1984       +8     
  Lines       77240    77441     +201     
  Branches     1343     1344       +1     
==========================================
+ Hits        67932    67938       +6     
- Misses       8903     9098     +195     
  Partials      405      405              
Flag Coverage Δ
integrationtests 64.14% <38.46%> (-0.04%) ⬇️
unittests 86.42% <27.65%> (-0.22%) ⬇️
Components Coverage Δ
pkg_aws_library 93.59% <ø> (ø)
pkg_celery_library 83.41% <ø> (ø)
pkg_dask_task_models_library 79.33% <ø> (ø)
pkg_models_library 93.08% <100.00%> (+<0.01%) ⬆️
pkg_notifications_library 85.20% <ø> (ø)
pkg_postgres_database 87.95% <ø> (ø)
pkg_service_integration 70.19% <ø> (ø)
pkg_service_library 70.90% <0.00%> (-1.63%) ⬇️
pkg_settings_library 90.19% <ø> (ø)
pkg_simcore_sdk 84.99% <ø> (-0.06%) ⬇️
agent 93.53% <ø> (ø)
api_server 91.93% <88.88%> (-0.02%) ⬇️
autoscaling 95.74% <ø> (ø)
catalog 92.36% <ø> (ø)
clusters_keeper 99.13% <ø> (ø)
dask_sidecar 91.82% <ø> (-0.56%) ⬇️
datcore_adapter 97.94% <ø> (ø)
director 75.81% <ø> (-0.09%) ⬇️
director_v2 90.90% <100.00%> (-0.09%) ⬇️
dynamic_scheduler 96.68% <ø> (ø)
dynamic_sidecar 90.43% <ø> (ø)
efs_guardian 89.62% <ø> (ø)
invitations 91.44% <ø> (ø)
payments 92.62% <100.00%> (ø)
resource_usage_tracker 92.34% <ø> (+0.21%) ⬆️
storage 86.67% <ø> (-0.09%) ⬇️
webclient ∅ <ø> (∅)
webserver 87.67% <68.42%> (-0.04%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 27ff515...cbf77c2. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mergify
Copy link
Contributor

mergify bot commented Sep 26, 2025

🧪 CI Insights

Here's what we observed from your CI run for cbf77c2.

✅ Passed Jobs With Interesting Signals

Pipeline Job Signal Health on master Retries 🔍 CI Insights 📄 Logs
CI system-tests Base branch is healthy, but retries were needed. Could be early signs of flakiness 👀 Healthy 1 View View
unit-tests Base branch is healthy, but retries were needed. Could be early signs of flakiness 👀 Healthy 2 View View

@pcrespov pcrespov force-pushed the fix/many-webservers-rpc branch 2 times, most recently from f738124 to cc41ab4 Compare October 1, 2025 08:17
@pcrespov pcrespov changed the title WIP 🎨: Fix/many webservers rpc WIP 🎨 New rpc client in the api-server to avoid cross-service calls with the web-server variants Oct 1, 2025
@pcrespov pcrespov changed the title WIP 🎨 New rpc client in the api-server to avoid cross-service calls with the web-server variants 🎨 New rpc client in the api-server to avoid cross-service calls with the web-server variants Oct 1, 2025
@pcrespov pcrespov marked this pull request as ready for review October 1, 2025 08:23
@pcrespov pcrespov force-pushed the fix/many-webservers-rpc branch from 8da9af1 to db68e64 Compare October 1, 2025 10:44
Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please consider my comment regarding casting, I think we can do better there.
Thanks!

@pcrespov pcrespov requested a review from GitHK October 1, 2025 12:55
@pcrespov pcrespov force-pushed the fix/many-webservers-rpc branch from fd4d912 to c8d44e5 Compare October 1, 2025 13:49
Copy link
Contributor

@wvangeit wvangeit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot @pcrespov

@pcrespov pcrespov force-pushed the fix/many-webservers-rpc branch from 64e97cc to bfa9879 Compare October 3, 2025 08:59
@sonarqubecloud
Copy link

sonarqubecloud bot commented Oct 3, 2025

@pcrespov pcrespov disabled auto-merge October 3, 2025 15:22
@pcrespov pcrespov merged commit b21d32f into ITISFoundation:master Oct 3, 2025
192 of 201 checks passed
@pcrespov pcrespov deleted the fix/many-webservers-rpc branch October 3, 2025 15:23
pcrespov added a commit that referenced this pull request Oct 3, 2025
…h the web-server variants (🚨) (#8431)

Co-authored-by: Mads Bisgaard <[email protected]>
pcrespov added a commit that referenced this pull request Oct 3, 2025
matusdrobuliak66 pushed a commit that referenced this pull request Oct 6, 2025
…h the web-server variants (🚨) (#8431)

Co-authored-by: Mads Bisgaard <[email protected]>
matusdrobuliak66 pushed a commit that referenced this pull request Oct 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

a:apiserver api-server service a:services-library issues on packages/service-libs a:webserver webserver's codebase. Assigning the area is particularly useful for bugs t:maintenance Some planned maintenance work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants